FIX Stop JSON retries for adversarial refusals - #2443
Conversation
Detect blocked and errored adversarial chat responses before schema parsing so terminal target failures do not consume the malformed-JSON retry budget. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
hannahwestra25
changed the title
Aren't they all using the same adv conv manager? |
yes updated the description! |
|
|
||
| def _parse(response: Message) -> AdversarialReply: | ||
| if response.is_error(): | ||
| raise BadRequestException( |
There was a problem hiding this comment.
Could we preserve the error type or status here? response.is_error() also covers errors such as processing and unknown, so an upstream server failure could become BadRequestException(400) and look like a client-side refusal. For example, we could use BadRequestException only for blocked responses and propagate another terminal exception for other error responses. Let me know if I misunderstood the intended normalization. If you think this is fine that's also ok.
2 participants
Description
OpenAI structured refusals arrive as blocked/error
Messages, but the shared adversarial conversation manager parsed them as malformed JSON and exhausted the JSON retry budget.This change detects errored messages before parsing and raises
BadRequestException, so TAP, Red Teaming, and Crescendo all stop retrying deterministic failures. TAP prunes the affected node; Red Teaming and Crescendo fail through the standard strategy lifecycle. Malformed normal responses still retry.Tests and Documentation
tyfor changed files.